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SUMMARY 


Problem 


To  Investigate  computer-administered  tests  as  a means  of  Improving 
predictability  of  on-Job  performance,  a set  of  computerized  and  paper- 
and-pencll  tests  measuring  five  job-relevant  personal  attributes  were 
developed.  These  attributes  are:  (1)  Short-term  Memory,  (2) 

Perceptual  Speed,  (3)  Perceptual  Closure,  (4)  Movement  Detection, 
and  (5)  Dealing  with  Concepts/Information.  This  report,  the 
second  of  two,  presents  data  from  several  related  studies  concerning 
the  validity  and  Job-related  characteristics  of  the  experimental 
battery. 


Approach 

Questionnaires  mailed  out  9 to  10  months  after  the  original  test 
administration  were  used  to  collect  supervisory  ratings  of  on-job 
performance  of  research  subjects.  The  usefulness  of  34  experimental 
and  operational  variables  for  predicting  both  global  performance  and 
performance  on  42  Individual  job  elements  was  Investigated  for 
personnel  In  the  Electrician's  Mate,  Personnelman,  Sonar  Technician, 
and  Apprenticeship  rating  groups.  Correlational  and  multiple- 
regression  analyses  were  carried  out. 

The  study's  objectives  are  listed  below: 

1.  To  compare  validities  of  the  experimental  variables  with 
similar  validities  for  operational  classification  tests. 

2.  To  determine  the  consistency  of  predictive  relationships 
of  tests  for  job  elements  across  ratings. 

3.  To  test  the  knowledgeablllty  of  personnel  experts  concerning 
the  relative  Importance  of  particular  personal  attributes  for  the 
performance  of  particular  job  elements. 

4.  To  determine  the  usefulness  of  ratings  of  Importance  and 
percentage  of  time  spent  on  job  elements  as  moderator  variables  for 
predicting  global  performance  marks  from  job  element  marks. 

5.  To  compare  the  predictive  accuracy  of  synthetic  validity 
with  the  widely  used  stepwise  multiple-regression  technique. 
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Findings  and  Conclusions 


1.  About  10  percent  of  the  zero-order  validities  of  experimental 
tests  were  statistically  significant,  with  most  of  the  significant 
validities  being  for  Sonar  Technician.  No  experimental  test  was 
significant  for  a personal  attribute  in  more  than  one  rating.  Most 
experimental  tests  with  significant  validities  were  computer- 
administered. 

2.  For  all  three  Technical  ratings,  selected  experimental  tests 
substantially  enhanced  the  predictability  of  the  operational  battery. 
Of  the  increases  in  the  shrunken  multiple-regression  coefficients, 
that  for  Sonar  Technician  appeared  to  be  operationally  the  most 
useful. 


3.  Of  the  five  attributes,  those  for  which  computerized  measure- 
ment would  be  the  most  useful  for  improving  predictability,  and  would 
add  the  most  to  the  predictability  of  on-job  performance,  were 
Movement  Detection  and  Dealing  with  Concepts/ Informat ion. 

4.  There  was  little  or  no  evidence  of  consistency  of  job  element 
characteristics  across  ratings.  The  variables  having  significant 
zero-order  validities  for  a job  element  in  one  rating,  in  general, 
did  not  have  significant  validities  for  the  same  element  in  ottier 
ratings. 


5.  The  job  elements  which  were  highly  predictable  were  those 
which  were  important  and  central  to  the  duties  of  particular 
ratings.  Thus,  for  Personnelman,  the  highly  predictable  job 
elements  involved  skills  in  writing,  verbal  communication, 
influencing  others,  and  communicating  routine  information.  For 
Sonar  Technicians,  skills  with  visual  displays  were  highly 
predictable. 

6.  For  Technical  ratings,  the  most  effective  predictors  of 
job  element  marks  were  experimental  tests,  with  the  best  such 
tests  being  computer-administered.  For  the  Apprenticeship  group, 
the  most  effective  predictors  were  operational  tests. 

7.  In  the  Technical  ratings,  slight  increases  in  accuracy  of 
predicting  global  from  job  element  marks  were  associated  with 
using  Time  Required  and  Importance  ratings  as  moderators.  However, 
the  increases  appeared  to  be  a result  of  capitalization  on  chance 
relationships  and  were  not  of  practical  importance. 
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8.  Generally  low  correlations  were  found  between  empirically- 
derived  estimates  of  Importance  of  personal  attributes  for 
particular  job  elements  and  similar  estimates  based  on  the  judgments 
of  personnel  experts.  These  findings  imply  that  personnel  experts 
cannot,  in  the  abstract,  make  accurate  determinations  of  attribute 
requirements  for  job  elements. 

9.  Synthetic  validity  was  generally  not  as  accurate  as  multiple 
regression  for  predicting  job  performance  . 


Recommendations 


1.  Because  measures  of  Closure  had  significant  validities  for 
job  performance  in  all  four  ratings/rating  groups,  and  had  both 
positive  and  negative  beta  weights,  a Closure  test  would  be  highly 
suitable  for  a battery  designed  to  predict  on-job  performance. 

Measures  of  Closure  should  be  added  to  any  classification  battery 
designed  to  predict  the  on-job  performance  of  Navy  enlisted 
personnel,  (p.  10) 

2.  Skills  in  processing  and  interpretation  of  visual  information, 
in  addition  to  those  measured  by  the  operational  tests,  appear  to  be 
important  for  the  on-job  performance  of  Sonar  Technicians.  Measure- 
ment of  these  skills  by  experimental  variables  in  the  present  research 
resulted  in  Increases  in  predictability  substantial  enough  to  provide 
practical  improvements  in  the  selection  of  Sonar  Technicians.  However, 
before  these  experimental  tests  can  be  used  operationally,  the  study 
must  be  replicated  for  at  least  100,  and  preferably  200,  Sonar 
Technician  personnel.  In  addition,  testing  should  be  broadened  to 
include  personnel  in  other  ratings  requiring  substantial  processing 

of  visual  information  from  consoles  or  scopes  (such  as  Operations 
Specialist),  (pp.  9-11,  14,  16) 
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BACKGROUND  AND  PURPOSE 


Computerized  testing  equipment,  in  addition  to  its  capabilities 
for  adaptive  testing  (Cleary,  Linn,  & Rock,  1968;  Wood,  1974;  see 
especially  Weiss  & Betz,  1973),  is  promising  as  a means  of  increasing 
the  scope  of  measurement.  Such  equipment  permits  exact  determination 
of  stimuli  exposure  times,  control  of  moving  stimuli,  exact  control 
of  the  sequence  of  stimuli  presentation,  successive  modification  of 
stimuli  components,  determination  of  response  latencies,  and 
sophisticated  classification  and  scoring  of  responses.  Since 
these  characteristics  are  not  available  for  paper-and -pencil  tests, 
the  use  of  computer-administered  tests  may  substantially  increase 
the  number  of  abilities  which  can  be  used  to  predict  personnel 
performance  on  the  job. 

This  is  the  second  report  on  a study  which  involved  the  develop- 
ment and  validation  of  a set  of  computer-administered  tests  of 
position-relevant  personal  abilities  or  attributes.  These  attributes 
are:  (1)  Short-term  Memory,  (2)  Perceptual  Speed,  (3)  Perceptual 

Closure,  (4)  Movement  Detection,  and  (5)  Dealing  with  Concepts/ 
Information.  The  initial  report  (Cory,  1974)  described  the  study 
rationale  and  the  interrelationships  of  the  computerized  and  non- 
computerized (paper-and-pencll)  tests  designed  to  measure  the  same 
abilities. 

The  present  report  has  the  following  objectives: 

1.  To  compare  the  validities  of  the  experimental  variables 
for  global  job  performance  with  similar  validities  for  operational 
classification  tests. 

2.  To  determine  the  consistency  of  predictive  relationships 
of  tests  for  job  elements  across  ratings — that  is,  the  extent  to 
which  performance  on  a job  element  was  predicted  by  the  same 
variables  from  rating  to  rating. 

3.  To  test  the  knowledgeability  of  personnel  experts  concerning 
the  relative  Importance  of  particular  personal  attributes  for 
performing  particular  job  elements. 

4.  To  compare  the  predictive  accuracy  of  the  synthetic-validity 
technique  with  the  widely  used  stepwise  multiple-regression  technique. 

5.  To  determine  the  usefulness  of  ratings  of  importance  and 
percentage  of  time  spent  on  job  elements  as  moderator  variables  for 
predicting  global  performance  marks  from  job  element  marks. 
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PROCEDURE 


Sample 

For  the  previous  stage  of  this  research,  a battery  of  17  experi- 
mental tests  (eight  computerized  and  nine  paper-and-pencil)  was 
developed  for  experimental  administration  in  conjunction  with  nine 
operational  tests  which  are  routinely  given  to  Navy  enlisted 
personnel.  The  experimental  tests  were  administered  during  May  and 
June  of  1972,  to  385  enlisted  personnel  at  the  Naval  Training 
Center  (NTC) , San  Diego.  Subjects  were  either  in  their  first  2 weeks 
of  "A"  school  training  (at  the  Service  School  Command)  for  rating  as 
(1)  Electrician's  Mate  (EM),  (2)Personnelman  (PN) , or  (3)  Sonar 
Technician  (ST),  or  in  their  final  week  of  recruit  training.  Subjects 
in  the  latter  category  were  limited  to  the  upper  50  percent  of  the 
ability  distribution  for  Navy  enlisted  personnel,  and  thus  were 
comparable  in  general  mental  ability  to  those  who  had  received  "A" 
school  assignments.  As  it  turned  out,  most  of  the  personnel  in  the 
group  were  assigned  to  general  apprenticeship  ratings  rather  than  to 
"A"  school  training.  For  purposes  of  this  report,  they  were  classi- 
fied as  an  Undifferentiated  Apprenticeship  (UA)  group. 

It  has  been  suggested  that  supervisory  marks  based  on  job  elements 
would  provide  more  reliable  and  valid  measures  of  performance  than 
the  commonly  used  global  marks  (Lawshe,  1952;  Guion,  1965;  Mecham  & 
McCormick,  1969).  Thus,  for  the  present  report,  about  10  months 
after  the  experimental  testing  (after  subjects  had  served  in  the 
fleet  for  several  months),  a mailout  questionnaire  was  used  to 
collect  supervisory  ratings  covering  both  global  and  job  element 
aspects  of  on-job  performance.  The  22-9 subjects  for  whom  supervisory 
marks  were  returned  were  Included  in  the  analyses  reported. 


Variables 


The  variables  used  were  experimental  (computerized  and  paper- 
and-pencil)  and  operational  test  scores,  selected  biographical 
variables,  and  measures  of  job  performance.  Usually,  test  scores 
consisted  of  the  number  of  correct  answers.  However,  scores  of 
tests  having  multiple-choice  or  true-false  items  were  corrected 
for  guessing. 


Computerized  Tests.  The  research  was  conducted  on  an  IBM  1500 
system  with  cathode  ray  tube  display  units  and  screens  for  film 
presentation,  linked  on-line  to  an  IBM  1130  computer.  A general 
description  of  the  eight  computerized  experimental  tests  is  Included 
in  Cory  (1974).  These  tests,  plus  four  more  which  were  derived  from 
special  capabilities  of  computerized  testing  equipment,  are  JLlsted 
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in  Table  1.  The  names  shoxm  for  a few  of  the  variables  are  slightly 
different  from  those  which  have  been  previously  reported  (Cory, 

1974).  These  changes  were  made  to  simplify  comprehension  and 
provide  greater  ease  of  reference.  Also,  in  regard  to  derived  tests, 
attention  is  directed  to  the  fact  that  CLO-LAT  differs  from  the  WORD- 
LAT  and  FIG-LAT  latency  measures  in  that  it  measures  time  required  to 
correctly  identify  an  object  rather  than  speed  of  response.  Error 
rates  did  not  influence  the  latter  two  latency  scores. 


Noncomput er iz ed  (paper-and-pencil)  tests.  The  noncomputerized 
tests  are  also  described  in  Cory  (1974).  They  are  listed  in  Table  2. 
Most  of  these  tests  are  from  the  Educational  Testing  Service  (ETS) 
factor  reference  battery  (French,  Ekstrom,  & Price,  1963). 


Biographical  variables.  Extracted  from  military  records  for 
use  as  biographical  variables  were  Years  of  Education  (YRED)  and 
Year  of  Birth  (YRBI) . 


Operational  tests.  The  operational  tests  are  also  Included  in 
Table  2.  Scores  for  these  tests  were  extracted  from  the  Enlisted 
Master  Tape  Record  (EMTR) . Scores  for  the  Armed  Forces  Qualification 
Test  (AFQT)  are  recorded  as  percentiles,  and  normally  range  from  10 
to  99.  The  tests  in  the  Navy  enlisted  classification  battery  (the 
last  eight  listed)  are  scored  as  Navy  standard  scores  with  means  for 
a Navy  population  of  approximately  50,  and  standard  deviations  of 
approximately  10. 


Measures  of  job  performance.  An  adaptation  of  the  Position 
Analysis  Questionnaire,  a broad-based  empirically-derived  instrument 
which  has  been  extensively  used  for  job-classification  research  by 
E.  J.  McCormick  and  his  associates  (see  McCormick,  Jeanneret,  & 
Mecham,  1972)  was  used  to  collect  the  ratings  of  job  performance. 

To  adapt  the  form,  three  committees  from  NTC,  San  Diego,  consisting 
of  chief  petty  officers  in  each  of  the  three  technical  ratings, 
were  convened  and  asked  to  select  job  elements  in  the  Position 
Analysis  Questionnaire  which  were  present  in  the  rating.  During 
this  step  the  wording  of  the  job  element  was  simplified,  wherever 
possible.  There  was  high  agreement  among  members  of  each  panel 
concerning  the  job  elements  which  were  appropriate  to  be  included 
in  the  questionnaire. 
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TABLE  1 


Computerized  Experimental  Teats 


Test 

Acronym 

Content  Area 

Conventionally  Derived 

Memory  for  Objects 

Short-term  recall  of  groups 
of  pictures 

Memory  for  Words 

Short-term  recall  of  groups 
of  words 

Memory  for  Numbers  (Visual) 

Short-term  recall  of  groups 
of  digits  presented 
visually 

Comparing  Figures 
Machine-paced 
Self-paced 

Comparison  of  objects  for 
similarities  and 
differences  (a  modified 
test  of  perceptual  speed) 

Recognizing  Objects 

Perceptual  closure  in  the 
recognition  of  objects 
from  fragmentary  details 

Memoxry  for  Patterns 
True-false 
Free  response 

Short-term  memory  for 

patterns  formed  by  moving 
stimuli 

Twelve  Questions 

Convergent  reasoning  using 
information  presented  in 
a "Twenty  Questions" 
format 

Password 

Convergent  reasoning  on  the 
basis  of  information 
presented  in  a "Password" 
format 

Derived  from  Special  Capabilities  of  Computerized  Testing  Equipment 

Latency  of  word  memory 
responses 

WORD-LAT 

Total  elapsed  time  in 

seconds  before  initiation 
of  the  subject *8  responses 
for  Memory  for  Words 

Latency  of  comparing 
figures  responses 

FIG-LAT 

Total  number  of  seconds 
which  elapsed  prior  to 
initiation  of  subject's 
responses  for  Comparing 
Figures , self-paced 

Total  time  required  for 
identification  of  the 
objects  in  the  com- 
puterized closure  test 

CLO-LAT 

Total  viewing  time  in 
seconds  elapsing  before 
all  objects  were 
identified 

Magnitude  of  pattern  error 

PAT-ERR 

A measure  of  the  total 
extent  to  which  the 
response  pattern  failed 
to  duplicate  the  stimuli 
in  Memory  for  Patterns, 
free  response 
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TABLE  2 


Noncomputerized  Experimental  and  Operational  Tests 


Test 

Acronym 

Content  Area 

Experimental 

Memory  for  Numbers  (Auditory) 

Short-term  recall  of  groups  of 
digits 

Object  Number^ 

Short-term  associative  memory 

Counting  Numbers 

Scanning  rows  of  digits  to 
Identify  specified  numbers 
and  count  their  frequencies 

Gestalt  Completion^ 

Perceptual  closure  In  the 
recognition  of  objects  from 
fragmentary  details 

Concealed  Words^ 

Perceptual  closure  In  the 
recognition  of  words  from 
fragmentary  details 

Hidden  Patterns® 

Inspection  of  complex  patterns 
to  recognize  simple  target 
patterns 

Drift  Direction 

Detection  of  the  direction  of 
minimal  movements 

Nonsense  Syllogisms® 

Syllogistic  reasoning  using 
nonsense  words 

Inference® 

Evaluation  of  the  correctness 
of  Inferences  from  given 
statements 

Operational 

Armed  Forces  Qualification 
Test 

AFQT 

Vocabulary^  arithmetic 

reasoning,  spatial  reasoning, 
and  mechanical  knowledge 

General  Classification  Test 

GCT 

Word  meanings  and  the  ability 
to  reason  verbally 

Arithmetic  Reasoning  Test 

ART 

Quantitative  aptitude,  Including 
mathematical  reasoning  and 
problem  solving 

Mechanical  Test 

MECH 

Basic  mechanical  and  electrical 
knowledge  and  mechanical 
principles 

Clerical  Test 

CLER 

Perceptual  speed  and  accuracy 

Sonar  Pitch  Memory  Test 

SONR 

Ability  to  perceive  and 

remember  small  differences 
In  tonal  pitches 

Radio  Code  Aptitude  Test 

RADO 

Ability  to  learn,  remember, 
and  use  sound  patterns  as 
symbols 

Electronic  Technician 
Selection  Test 

ETST 

Mathematics,  science, 

electricity,  and  electronics 

Shop  Practices 

SHOP 

Knowledge  of  tools  and  shop 
equipment 

^Factor  reference  test,  Kit  of  Reference  Tests  of  Cognitive  Factors. 
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Items  describing  42  job  elements,  together  with  estimates  of 
percentage  of  time  spent  and  Importance  of  each  element  for  global 
performance,  were  Included  in  the  questionnaire  (see  the  Appendix). 

In  addition,  a rating  of  global  performance  was  Included.  All  ratings 
were  carried  out  on  a 5-point  scale,  with  the  choices  designated: 

(1)  lowest  5 percent,  (2)  next  lowest  25  percent,  (3)  middle  40 
percent,  (4)  next  highest  25  percent,  and  (5)  highest  5 percent. 


Analysis 

Zero-order  validities  for  both  global  and  job  element  criteria 
were  computed  separately  for  the  four  rating  groups  and  were 
utilized  to  compute  multiple-regression  validities  by  means  of  a 
stepwise  predictor  selection  technique. 

Comparisons  were  concerned  with  the  extent  to  which  (1)  the 
experimental  variables  could  increment  the  predictive  accuracy 
achievable  from  operational  variables,  and  (2)  predictive  relation- 
ships were  consistent  for  job  elements  across  ratings.  To  test  the 
accuracy  of  expert  judgments  concerning  the  validities  of  personal 
attributes  for  performance  of  job  elements,  the  validities  actually 
found  were  compared  with  predicted  validities  from  Mecham  and 
McCormick  (1969) . Comparisons  of  the  synthetic-validity  technique 
with  multiple  regression  for  a global  criterion  were  carried  out 
using  a modification  of  the  methodology  used  in  a study  by  Gulon 
(1965) . The  predictability  of  global  performance  marks  from 
composites  of  job  element  marks  was  investigated  using  Time  Spent 
and  Importance  marks  as  weighting  variables  in  the  following  designs: 
(1)  raw  supervisory  marks,  (2)  linear  prediction,  an^  (3)  moderated 
prediction.  The  latter  two  methods  were  defined  by  Saunders  (1956) . 


RESULTS 


Incidence  of  questionnaire  returns  and  final  sample  sizes  of  the 
rating  groups  are  shown  in  Table  3.  After  preliminary  analysis  of 
the  returned  questionnaires,  a 22-item  subset  of  the  job  elements 
having  the  largest  sample  sizes  was  selected  for  analysis.  These 
job  elements,  designated  the  common  job  elements,  are  shown  in 
Table  4,  together  with  their  subsample  sizes. 
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TABLE  3 


Sample  Sizes  for  Test  Administration  and 
Performance  Follow-up 


Rating  Test  Admin- 

Group  Istratlon 

Performance 

Follow-up 

Unavailable 
Because  of 
Transfer, 
School  Assign- 
ment, Discharge, 
etc. 

Not 

Returned 

Personnelman 

(PN) 

61 

54 

4 

3 

Electrician's 
Mate  (EM) 

77 

27 

33 

17 

Sonar  Technician 
(ST) 

45 

37 

5 

3 

Recruits  in 

172 

111 

17 

upper  half  of 
the  recruit 
distribution  of 
ability  (classi- 
fied as  Undif- 
ferentiated 
Apprenticeships) 
(UA) 

TOTALS 

355® 

229 

86 

40 

thirty  cases  were  dropped  because  of  Incomplete  experimental 
data,  bad  records,  or  rating  mlsclasslflcation. 
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TABLE  4 


Sample  Sizes  for  the  22  Most  Common  Job  Elements 


Job  Element 

EM 

Rating  Group 
PN  ST 

UA 

Using  Written  Materials 

48 

30 

71 

Using  Pictorial  Materials 

20 

— 

32 

— 

Using  Visual  Displays 

— 

— 

35 

66 

Using  Spoken  Verbal  Communication 

20 

52 

36 

92 

Using  Nonverbal  Sounds 

— 

— 

31 

— 

Analyzing  Information 

20 

— 

— 

— 

Compiling  Data 

— 

49 

— 

— 

Manual  Control,  Nonprecision  Tools 

27 

— 

— 

80 

Manual  Control,  Precision  Tools 

23 

— 

— 

— 

Operating  Keyboard  Devices 

— 

53 

— 

— 

Adjusting  Machines/Equipment 

23 

— 

29 

— 

Assembling-Disassembling 

23 

Hand  and  Arm  Manipulation/Coordination 

22 

Hand  and  Ear  Coordination 

— 

— 

31 

— 

Persuading/Influencing  Others 

— 

40 

— 

— 

Exchanging  Routine  Information 

— 

51 

— 

69 

Unusually  Good  Precision 

— 

— 

29 

69 

Attention  to  Details,  Completing  Work 

25 

51 

36 

102 

Vigilance,  Continually  Changing  Details 

20 

Coping  with  Time  Pressures 

22 

49 

— 

78 

Working  with  Distractions 

— 

48 

— 

84 

Keeping  up-to-date 

52 

30 

86 
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Prediction  of  Global  Criteria 


Zero-order  correlations.  Zero-order  correlations  of  the  experi- 
mental variables  with  supervisors’  ratings  of  global  job  performance 
are  shown  In  Table  5.  Nine  of  the  92  validity  coefficients  were 
statistically  significant  (lOpercent),  five  of  which  were  for 
computerized  tests.  No  experimental  test  was  significant  for  more 
than  one  rating.  Most  of  the  significant  validities  were  for  STs, 
with  fewer  numbers  of  significant  validities  for  EMs,  PNs,  and  UAs. 

In  comparison,  five  of  36  validities  of  the  operational  tests 
were  statistically  significant  (Table  6),  three  of  which  were  for 
the  UA  group.  Thus,  the  operational  battery  was  best  for  predicting 
global  performance  in  apprenticeship  ratings,  whereas  the  experimental 
battery  was  more  useful  for  predicting  performance  in  technical  ratings, 
and  was  particularly  good  for  predicting  the  performance  of  STs. 

The  coefficients  suggest  that,  of  the  five  attributes,  the  most 
promising  for  computerized  measurement  are  Movement  Detection  and 
Dealing  with  Concepts/Information. 


Multiple  correlations.  Multiple  regression  statistics  for 
optimal  sets  of  the  operational  and  experimental  variables  are 
shown  In  Tables  7 through  10.  The  optimum  batteries  were  determined 
by  means  of  a predictor  selection  technique  using  the  accretion  method. 
Predictor  selection  was  terminated  by  reference  to  a conservative 
decision  rule!  F ratios  were  used  to  determine  the  statistical 
significance  of  the  regressions. 


^The  following  termination  criteria  were  used: 

1.  To  be  added  to  the  battery,  a variable  must  increase  not  only 
the  multiple-correlation  coefficient,  but  also  the  estimated  cross- 
validated  correlation  coefficient. 

2.  In  general,  variables  added  to  the  battery  must  have  alpha 

values  significant  at  £<  .05.  However,  the  alpha  value  of  a 
variable  could  be  in  the  range  .05<  < .10,  provided  that  the 

next  variable  added  to  the  battery  was  significant  at  £^<.05  and 
the  alpha  value  of  the  previous  variable  dropped  to  £<.05  on  the 
new  selection  step.  In  other  words,  predictor  selection  for  the 
battery  was  stopped  at  the  first  step  for  which  a hypothesis  of 

no  increase  in  R could  not  confidently  be  rejected  at  £ < .05. 
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TABLE  5 


Zero-order  Validities  of  Experimental  Variables 
for  Global  Performance 


Rating  Group 

Predictor 

EM 

PN 

ST 

UA 

(N  - 27) 

(N  - 54) 

(N  - 37) 

(N  = 111) 

Short-term  Memory 

Object  Number 

-.26 

.13 

-.03 

-.01 

Memory  for  Objects^ 

-.16 

-.03 

-.05 

-.07 

Memory  for  Words*^ 

-.33 

.20 

.13 

.01 

Memory  for  Numbers 

(Visual) c -.15 

.20 

.38* 

-.01 

Memory  for  Numbers 

(Auditory)  -.15 

.17 

.22 

.08 

Perceptual  Speed 

Counting  Numbers 

.03 

.04 

.42* 

.06 

Comparing  Figures,  ^ 

Machine- 

paced^ 

.02 

-.10 

.07 

-.06 

Comparing  Figures, 

Self-paced*^  .06 

.07 

.21 

.08 

Closure 

Gestalt  Completion 

-.28 

-.26* 

.28 

.06 

Concealed  Words 

-.37* 

-.14 

.13 

-.10 

Hidden  Patterns 

-.04 

.23 

.33* 

.11 

Recognizing  Objects 

-.11 

-.06 

.25 

-.05 

Movement  Detection 

Drift  Direction 

-.29 

.07 

.02 

.06 

Memory  for  Patterns 

, True- 

false^ 

.15 

-.07 

.42* 

.07 

Memory  for  Patterns 

, Free 

Response^ 

.19 

.21 

.23 

.19* 

Dealing  with  Concepts/Information 

Nonsense  Syllogisms 

-.30 

.01 

.30 

-.06 

Inference 

.18 

.19 

.00 

.13 

Twelve  Questions^ 

-.20 

.28* 

.21 

.11 

Password^ 

.08 

.13 

.33* 

.04 

Special  Variables 

WORD-LAT^ 

-.24 

-.06 

-.05 

-.11 

CLO-LAT^ 

.05 

.02 

-.24 

-.11 

FIG-LAT*^ 

-.04 

.00 

.02 

.04 

PAT-ERR*^ 

-.24 

-.17 

-.26 

-.13 

*Signif leant  at  2.  ^ ‘OS. 

Q 

Computer-administered  test 


TABLE  6 

Zero-order  Validities  of  Operational  Variables 
for  Global  Performance 


Rating  Group 

Predictor 

EM  PN  ST 

UA 

(N  “ 21)  (N  ” 31)^(N  - 35)  (N  = 109)*^ 

Paper-and-Pencil  Tests 


AFQT 

-.09 

.15 

-.12 

.13 

GCT 

.01 

.24 

.11 

.07 

ARI 

-.20 

.10 

.38* 

.25** 

MECH 

.04 

.23 

-.04 

-.12 

CLER 

.21 

-.15 

.11 

.19* 

SONR 

-.08 

.15 

-.08 

-.03 

RADO 

-.06 

.11 

.15 

.15 

ETST 

.16 

.31 

-.09 

.33** 

SHOP 

.20 

.38* 

-.21 

.17 

Biographical  Variables 

YRBI 

-.12 

.06 

.01 

-.11 

YRED 

.11 

.05 

-.02 

.22* 

Complete  data-were  not  available  for  some  of  the  tests. 


*Signif leant  at  £ < .05. 

**Signif leant  at  £ < .01. 


TABLE  7 


Optimal  Predictive  Composites  for  Global  Performance 
of  Electrician's  Mates 


Predictor  Set 

Weight 

R 

Expected  Cross- 

Beta  Weight  in 

Determination 

validation 

Predictor 

Final  Composite 

N 

Operational 

Classification 


Test  Scores 

.21 

.00 

CLER 

27 

Complete  Set  of 

.37 

.00 

Concealed  Word 

-.40 

27 

Experimental  and 

.49 

.20 

CLER 

.39 

Operational 

.58 

.28 

Drift  Direction 

-.28 

Variables 

.65 

.34 

PAT-ERR 

-.50 

.71 

.40 

Memory  for  Words 

-.40 

.78 

.53 

YRBI 

-.36 

TABLE  8 


Optimal  Predictive  Composites  for  Global  Performance 

of  Personnelmen 


R 

Predictor  Set  Weight  Expected  Cross-  Beta  Weight  in 

Determination  validation  Predictor  Final  Composite  N 


Operational 

Classification 


Test  Scores 

.38 

.12 

Complete  Set  of 

.38 

.12 

Experimental  and 

.47 

.20 

Operational 

.64 

.46 

Variables 

.71 

.52 

.80 

.65 

.86 

.74 

SHOP 

• 

00 

30 

SHOP 

.22 

30 

Gestalt  Completion 

-1.19 

GCT 

1.40 

FIG-LAT 

.69 

WORD-LAT 

- .40 

Memory  for  Patterns, 
True-false 

.37 

TABLE  9 


Optimal  Predictive  Composites  for  Global  Performance 
of  Sonar  Technicians 


Predictor  Set 


R 

Weight  Expected  Cross- 

Determlnation  validation  Predictor 


Beta  Weight  in 
Final  Composite  N 


Operational 

Classification 


Test  Scores 

.38 

.22 

Complete  Set  of 

.42 

.28 

Experimental  and 

.54 

.40 

Operational 

Variables 

.61 

.46 

.66 

.50 

.73 

.58 

ARI 

.38 

37 

Counting  Numbers 

.33 

37 

Memory  for  Patterns, 
True-false 

.32 

Nonsense  Syllogisms 

.29 

Recognizing  Objects 

.33 

Gestalt  Completion 

.32 

TABLE  10 


Optimal  Predictive  Composites  for  Global  Performance 
of  the  Apprenticeship  Group 


R 


Predictor  Set 

Weight 

De  termina  t ion 

Expected  Cross- 
validation 

Predictor 

Beta  Weight  in 
Final  Composite 

N 

Operational 

Classification 
Test  Scores 

.33 

00 

CM 

• 

ETST 

.33 

111 

Complete  Set  of 

.33 

.28 

ETST 

.33 

111 

Experimental  and 

.37 

.29 

CLER 

.21 

Operational 

.41 

.32 

Concealed  Word 

-.19 

Variables 


Predictability  of  the  operational  battery  was  quite  low — barely 
significant  for  most  ratings — and  validities  were  lower  for  the 
technical  ratings  than  for  UAs.  On  the  other  hand,  quite  high 
correlations  were  computed  for  the  combined  experimental  and  oper- 
ational battery,  particularly  for  the  technical  ratings.  Increases 
in  the  estimated  cross-validation  coefficients  of  the  complete  set 
over  those  of  the  operational  set  were  .53,  .62,  and  .36  for  the 
EM,  PN,  and  ST  ratings,  respectively.  Although  none  of  the  zero- 
order  validities  of  the  special  variables  were  statistically  sig- 
nificant, several  of  them  served  to  increment  the  multiple 
correlations. 

The  better  Ells  were  characterized  by  good  perceptual  speed, 
greater  age,  and  poor  closure,  perception  of  movement,  and  memory 
skills.  Distinguishing  characteristics  of  the  better  PNs  tended 
to  be  good  technical  knowledge,  verbal  ability,  memory,  and  speed 
of  recall  of  words,  and  poor  closure  and  speed  of  recall  of  flgural 
stimuli.  Good  STs  generally  had  superior  skills  in  perceptual 
speed,  movement  detection,  closure,  and  dealing  with  concepts/ 
information.  The  better  UAs  had  relatively  good  technical 
knowledge  and  perceptual  speed,  and  relatively  poor  closure  skills. 

Since  the  sample  sizes  for  all  three  technical  ratings  were 
small,  these  findings  should  be  interpreted  cautiously.  Of  the 
several  variables  in  the  predictor  composites  which  had  negative 
beta  weights,  three  were  measures  of  perceptual  closure,  one  was 
a measure  of  memory,  and  another  three  (PAT- ERR,  YRBI,  and  WORD- 
LAT)  resulted  from  directional  artifacts  of  the  scaling.  Thus,  if 
PAT- ERR  were  redefined  as  accuracy,  year  of  birth  as  age  in  years, 
and  WORD-LAT  as  speed  of  response,  the  scales  would  be  reversed 
and  the  negative  weights  would  become  positive. 


Prediction  of  Performance  on  Job  Elements 

Since  Information  concerning  the  predictability  of  job  elements 
would  be  useful  for  the  development  of  new  tests  and  selection 
batteries,  the  validities  of  the  predictors  were  computed  for  job 
elements.  These  were  examined  critically  for  (1)  evidence  of 
statistically  significant  predictability,  and  (2)  the  consistency 
of  the  validities  across  ratings. 


Zero-order  correlations.  The  significant  validities  of 
operational  and  experimental  variables  for  the  12  most  predictable 
common  job  elements  are  shown  in  Tables  11  and  12,  respectively. 
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TABLE  11 


Significant  Zero^order  Validities  of  the  Operational 
Variables  for  12  CoMon  Job  Eleoients 


Job 

> Element 

Predictor 

Verbal 

Non- 

Routine 

Attent ion 

Variable 

Rating 

Written 

Pictorial 

Visual 

CoiBBiuni- 

Precision 

Adjusting 

Influencing 

Infor- 

Good 

to 

Working  with 

Materials 

Materials 

Display 

cation 

Tools 

Equipment 

Others 

mation 

Precision 

Details 

Distractions 

Up-to-Dalo 

APQT 

ST 

-33* 

— 

— 

— 

CCT 

PN 





50* 

^ _ 

36* 

UA 

— 

22* 

— 

— 

27* 

ARI 

ST 

49** 







UA 

— 

24* 

— 

— 

41* 

27* 

23* 

MECH 

PN 

— 

— 

— 

— 

55** 

— 

38* 

CLC& 

UA 

25* 

— 

20* 

— 

— 

29* 

30** 

26* 

SOMR 

PN 

UA 

-26* 

— 

— 

— 

— 

— 

— 

37* 

-26* 

EM 

— 

— 



— 

— 

-44* 

— 

— 

RAOO 

ST 

36* 

41* 

33* 

— 

— 

— 

37* 

— 

39* 

DA 

— 

22* 

— 

— 

CTST 

UA 

— 

— 

— 

28* 

24* 

SHOP 

PN 

— 

— 

45* 

— 

— 

42* 

— 

YRED 

UA 

— 

21* 

22* 

— 

— 

34** 

31** 

26** 

Cell  Na 


EM 

15 

16 

21 

18 

19 

PN 

26 

29 

27 

20 

28 

29 

27 

31 

ST 

29 

30 

33 

34 

27 

27 

34 

28 

UA 

69 

66 

90 

79 

67 

67 

100 

84 

Note.  Decimal  points  were  oaitted  froa  validity  coefficients. 

Coefficients  significant  at  £ < .05  and  £ < .01  have  been  identified  by  single  and  double  asterisks,  respectively. 
A blank  cell  Indicates  nonsignificant  validity. 

A double  hyphen  ( — ) indicates  missing  data. 


Table  12 


Significant  Zero-K>rdar  Valldlclea  of  Che  Experimental 
Varlablaa  for  12  Common  Job  Eiamanta 


Predictor 
Variable  Racing 

Job  Elem4 

nC 

Written 

Haterlala 

Pictorial 

Materlala 

Vlaual 

Dlaplay 

Verbal 

Coamunl* 

cation 

Non- 

Preclalon 

Toole 

Ad  juat Ing 
Equipment 

Influ- 

encing 

Ochera 

Routine 

Infor- 

mation 

Good 

Preclalon 

Attention 

to 

Detalla 

Working  with 
Dletracclona 

Up-to- 

Dace 

ObJ.  No. 

EM 

- 

- 

- 

- 

- 

-38* 

- 

- 

Kern.  Obj. 

9! 

-24* 

-53*» 

•• 

•• 

-?4» 

— 

Hem. 

Worde 

ST 

PN 

A2* 

29* 

Mem.  for 
Noe.  (V) 

PN 

ST 

-■ 

■■ 

4fi** 

_ _ 

— 

„ 

-- 

33* 

Hem.  for 
Noe.  (A) 

PN 

EM 

29# 

— 

42** 

— 

— 

— 

-42* 

— 

— 

Counting 

Nxjmbere 

UA 

ST 

- 

- 

EC** 

J9*  _ 

Comp. 

Fit*.. 

Machine^ 

paced 

UA 

PN 

25* 

- 

- 

20* 

- 

- 

- 

29* 

30** 

26* 

44** 

Ceac. 

Comp. 

IM 

- 

- 

-40* 

- 

- 

-42* 

- 

Hidden 

Patteme 

EH 

PH 

ST 

39* 

30** 

-36* 

- 

32** 

** 

- 

Rec.  Obja. 

ST 

45* 

- 

— 

— 

Mem.  for 
Fata..  Free 
Reaponee 

ST 

UA 

PN 

uo* 

- 

- 

- 

24* 

3B» 

26— 

2S* 

Honaenae 

Eyle. 

EM 

-38* 

Inference 

PN 

37* 

— 

— 

42** 

— 

— 

i»9** 

_3T* 

— 

29* 

Twelve 

Queatlona 

PN 

ST 

U2*e 

— 

44** 

— 

36* 

55** 

42** 

37* 

Paaaword 

PN 

ST 

33* 

— 

43* 

„ „ 

46* 

— 

30* 

WRD-UT 

EM 

- 

- 

-4l* 

- 

- 

- 

- 

- 

CLO-LAT 

ST 

-47** 

-44  *« 

— 

— 

— 

-37* 

PAT-CTR 

UA 

PN 

— 

-21* 

-3T* 

-23* 

riC-LAT 

PN 

ST 

UA 

34* 

- 

-36* 

- 

31** 

CaU  Ha 


IM 

20 

20 

— 57 

23 

25 

PN 

45 

48 

37 

47 

47 

44 

48 

ST 

29 

31 

34 

3€> 

29 

29 

36 

30 

UA 

71 

92 

i6 

^9 

69 

102 

84 

86 

Wota.  Decimal  polnta  vara  omitted  from  the  validity  coaf flclaota. 

Coafflclanta  algnlf leant  at  £ < .05  and  £ < .01  have  bean  Identified  by  aingla  and  double  aatarlaka,  raapactlvaly. 
A blank  Indlcatea  nonalgnlf leant  validity. 

A double  hyphen  (~)  Indlcatea  mieelng  data. 
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In  these  tables,  cells  not  containing  connnon  job  elements  for  a rating 
are  shown  with  double  hyphens,  cells  having  nonsignificant  validities 
have  been  left  blank,  and  predictor  variables  with  no  statistically 
significant  £*s  have  been  omitted. 

The  operational  tests  were  primarily  predictors  of  writing  and  var- 
bal  communication  skills  and  to  a lesser  extent  were  significant  pre- 
dictors of  precision  and  accuracy.  In  both  Table  11  and  Table  12  the 
large  number  of  significant  validities  for  Attention  to  Details  and 
Keeping  up-to-date  together  with  the  apparent  lack  of  patterns  across 
ratings  relating  tests  to  these  job  elements  suggest  that  these  elements 
served  as  generalized,  nonspecific  positive  evaluation  categories. 

Generally,  operational  variables  were  less  effective  for  predicting 
performance  on  job  elements  in  technical  than  in  apprenticeship  ratings. 
For  the  three  technical  ratings,  a total  of  only  16  correlations  were 
statistically  significant,  compared  with  21  significant  correlations 
for  UAs.  However,  the  significant  validities  of  operational  variables 
rather  frequently  appear  not  to  reflect  underlying  mental  abilities  or 
constructs  as  they  are  commonly  defined  (Guilford,  1967,  Messick,  1973). 
Thus,  ARI,  and  SHOP  were  found  to  be  significant  predictors  of  verbal 
skills;  MECH  and  SHOP  were  indicators  of  ability  to  Influence  others; 

RADO  significantly  predicted  ability  in  verbal  communication  and  in 
interpretation  of  pictorial  materials;  and  ETST  was  an  Indicator  of 
ability  to  communicate  routine  information. 

The  paucity  of  distinctive  predictor-job  element  relationships  for 
operational  variables  may  be  related  to  the  lack  of  accurate  dimensional 
definition  in  the  selection  and  classification  tests,  which  has  been 
pointed  out  previously  (Cory,  1971,  pp.9-12).  Thus,  the  specific 
operational  test-job  element  validities  which  were  statistically 
significant  seem  best  explained  as  resulting  from  a low  to  moderate 
correlation  of  "g"  (the  primary  content  of  the  classification  battery) 
(Cory,  1971,  1974)  with  job  performance,  together  with  chance 
variations  in  the  sample  subgroups. 

Predictor-job  element  construct  validities  were  more  logically 
defined  for  the  experimental  tests  (Table  12).  Thus,  short-term  memory 
variables  tended  to  predict  Writing  and  Verbal  Communication  for  PNs; 
Dealing  with  Concepts/Information  variables  tended  to  predict  Writing 
and  Verbal  Communication,  Influencing  Others,  and  Communicating 
Routine  Information  for  PNs;  and  the  special  variables  predicted 
Visual  Display  for  STs. 

There  was  considerable  variation  across  ratings  in  the  types  of  tests 
which  were  significantly  related  to  performance  on  the  job  elements.  For 
instance:  for  Written  Materials  the  significant  predictors  for  PNs  in- 
cluded primarily  memory  and  verbal  reasoning  variables;  for  STs  memory 
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and  perceptual  variables;  and  for  UAa  perceptual  variables.  For  Verbal 
Communication  for  PNs  significant  predictors  were  academic  learnings, 
perceptual  and  reasoning  skills,  and  memory;  and  for  UAs  academic  learn- 
ings, perceptual  skills  and  years  of  education.  Significant  predictors 
for  N6n-precision  Tools  for  EMs  included  closure  measures  and  for  UAs, 
memory  and  years  of  education.  For  Adjusting  Equipment,  EMs  required 
memory  and  speed  and  STs  required  verbal  reasoning.  These  and  other 
differences  across  ratings  in  the  content  of  tests  having  significant 
validities  indicate  that  the  same  job  element  frequently  varied  consid- 
erably from  one  rating  to  another. 

Multiple  correlations.  Data  for  the  multiple  correlations  of  the 
most  predictive  three-variable  batteries,  shown  in  Tables  13  and  lA, 
also  indicated  that  experimental  tests  were  better  predictors  of  job 
performance  than  were  operational  tests.  Sixty— six  of  the  variables 
entering  into  these  multiple  regressions  were  experimental  variables — 
more  than  three  times  the  21  operational  selections.  Biographical 
variables  were  selected  nine  times  and  AFQT,  three  times. 

Once  again,  predictors  for  a particular  job  element  tended  to  differ 
considerably  from  one  rating  to  another.  Even  when  a variable  was  se- 
lected for  the  same  job  element  in  two  different  ratings,  the  beta  weights 
frequently  differed  markedly  (cf..  Inference  as  a predictor  for  Written 
Materials  for  PNs  and  STs,  and  Gestalt  Completion  and  CLO-LAT  as  predic- 
tors of  Verbal  Communication  for  EMs  and  STs) . 

Job  elements  were  generally  not  as  predictable  as  global  performance. 
Of  the  ten  Rs  for  job  elements  for  UAs,  eight  were  lower  than  the  R for 
the  global  criterion.  For  the  EMs,  PNs,  and  STs,  respectively,  5 of  11, 

10  of  10,  and  8 of  10  of  the  ^s  for  job  elements  were  lower  than  that  for 
global  performance.  The  multiple  correlations,  as  did  the  zero-order 
correlations,  showed  the  operational  battery  to  be  more  useful  for  UAs 
and  the  experimental  battery,  for  the  technical  ratings. 

Predicatability  of  Global  Marks  from  Job  Element  Marks 

Regressing  the  global  job  performance  mark  for  a rating  on  marks  for 
its  common  job  elements  will  provide  information  on  the  job  element 
dimensionality  of  the  global  mark.  Thus,  the  extent  to  which  each  mark 
increments  the  shrunken  ^ indicates  the  Importance  of  performance  on  the 
job  element  for  global  performance. 

Predictabilities  of  global  performance  from  the  job  element  marks 
are  shown  in  Table  15.  The  maximum  expected  cross-validatilon  ^ for  glo- 
bal marks  in  each  rating  was  fairly  close  to  the  reliability  of  global 
marks  found  in  previous  studies  (Cory,  1975;  and  unpublished  studies). 

The  first  predictor  selected  for  each  rating  was  a generalized  measure 
of  performance.  For  most  ratings,  performance  on  rather  narrowly  defined 
job  elements  incremented  the  predictability  of  the  generalized  variables. 
Although  the  variability  of  duty  assignments  of  UAs  was  undoubtedly 
greater  than  that  of  a particular  technical  rating,  the  global  marks  of 
UAs  were  as  predictable  as  those  of  other  ratings. 
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TADLF  13 


Optimal  Predictive  Composites  for  Harks  of  Job  Clements 
Common  to  Two  or  More  Ratings 


EM 

PN 

R 

1 

Criterion 

Shrunken 

Shrunken 

Clement 

Weight 

Cross- 

Variable 

Beta 

Weight 

Cross- 

Variable 

Beta 

Developn. 

val. 

Selected 

Weight 

DovelOt>m. 

val . 

Selected 

Weight 

.37 

.24 

Inference® 

.43 

Skill  Using 

.50 

.38 

Gestalt- 

-.46 

Written 

Completion® 

Materials 

.58 

.46 

Memory  for 

.33 

Numbers  (V)® 

Using 

Pictorial 

.44 

.05 

Recognizing 

-.44 

Materials 

Objects® 

Using 

Visual 

Displays 

Using  Spoken 
Verbal 

Conmunlcatlon 

.41 

.67 

.79 

.00 

.49 

.64 

MORD-LAT' 

Hidden  Pattern* 
CLO-LAT® 

-.58 

-.62 

-.42 

.48 

.56 

.40 

.46 

Memory  for 
Ntrabers  (A)® 
Password® 

.37 

.31 

Manual  Control, 
Nonpreclslon 
Tools 

.40 

.52 

.15 

.27 

Gestalt 

Completion® 

SHOP 

-.24 

.45 

.65 

.45 

GCT 

-.44 

.53 

.37 

Memory  for 

-.58 

Adjusting 

Objects® 

Machines/ 

.69 

.55 

Recognising 

-.42 

Equipment 

Objects® 

.76 

.62 

WORD-LAT® 

-.33 

.55 

.47 

12  Questions'^ 

.63 

Exchanging 

.60 

.51 

Memory  for 

.29 

Rout Ine 

Patterns, 

Information 

.66 

.55 

True-false® 
Concealed  Word* 

-.28 

Unusually 

Good 

Precision 

.42 

.18 

Gestalt 

-.37 

.33 

.18 

YRBI 

.50 

Attention 

Completion® 

to  Details 

.58 

.37 

Memory  for 

-.40 

.49 

.33 

12  Questions® 

.44 

Completing 

Numbers  (Af 

Work 

.68 

.48 

Nonsense 

-.35 

,59 

.39 

RADO 

.32 

Syllogisms  ® 

.46 

.31 

SHOP 

.89 

Coping  with 
Time  Pressure 

.55 

.79 

.39 

.71 

Recognizing 

Objects® 

Gestalt 

-.73 

-.60 

.61 

.72 

.47 

.60 

AFqrr 

Object  No.® 

-.64 

.43 

Completion® 

Working 

.37 

.25 

PAT-ERR® 

-.18 

with 

.48 

.32 

12  Questions® 

.48 

Distractions 

.58 

.38 

MECH 

-.35 

Keeping  Up- 

.34 

.21 

Hidden 

.34 

to-Date 

Patterns® 
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TABI£  13  (continued) 


Criterion 
El ement 

St 

UA  

R 

Variable 

Selected 

Beta 

Weight 

R 

Variable 

Selected 

Beta 

Weight 

Weight 

Developm. 

Shrunken 

Cross- 

val. 

Weight 

Developm. 

Shrunken 

Cross- 

val. 

Skill  Using 
Written 
Materials 

.49 

.68 

.78 

.35 

.57 

.68 

ARI 

AFQT 

Inference® 

.97 

-.53 

-.43 

.26 

.36 

.11 

.22 

SONR 

Comparing 

Figures® 

-.30 

.25 

Using 

Pictorial 

Materials 

.48 

.63 

.72 

.34 

.51 

.60 

CLO-LAT® 

RADO 

Password® 

-.52 

.37 

.35 

Using 

Visual 

Displays 

.44 

.56 

.64 

.31 

.42 

.49 

CLO-LAT^ 

Gestal r 

Completion® 
Memory  for 
Numbers  (V) ® 

-.44 

.41 

.32 

.23 

.00 

Counting 

Numbers® 

.23 

Using  Spoken 
Verbal 

Communication 

.33 

.06 

RADO 

.33 

.24 

.12 

ARI 

.24 

Manual  Control, 
Nonprecision 
Tools 

.24 

.32 

.10 

.17 

Memory  for 
Objects* 
YF.EI 

-.23 

.21 

Adjusting 

Machines/ 

Equipment 

.46 

.62 

.30 

.47 

Password^ 

CLO-LAT® 

.51 

-.41 

Exchanging 

Routine 

Information 

.41 

.47 

.52 

.35 

.39 

.43 

ARI 

Ynrx 

Concealed  Word® 

.39 

.24 

-.23 

Unusually 

Good 

Precision 

.36 

.00 

AFQT 

-.36 

.31 

.42 

.20 

.31 

YI^ 

FIG-LAT® 

.28 

.28 

Attention 
to  Details 
Completing 
Work 

.37 

.18 

RADO 

.37 

.30 

.36 

.39 

.23 

.27 

.29 

CLER 

YRED 

Memory  for 
Objects  ® 

.25 

.19 

-.17 

Coping  with 
Time  Pressure 

.26 

.34 

.13 

.21 

YRTTj 

Nonsense 

Syllogisms  ® 

.30 

-.23 

Working 

with 

Distractions 

.24 

.35 

.11 

.24 

Memory  for 
Objects  ® 
Memory  for 
Numbers  (V) ® 

-.28 

.26 

Keeping  Up- 
to-date 

.39 

.51 

.16 

.29 

Counting 
Numbers  ® 

12  Questions® 

.35 

.33 

.28 

.43 

.19 

.36 

Memory  for 
Patterns , 

Free  Response 
SONR 

.36 

e 

-.34 

^Experimental  variable. 
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TABIJP  l4 


Optimal  Predictive  Composites  for  Marks  of  ConsK^n 
Job  Elements  Unique  to  One  Rating 


EM 

PN 

ST 

R 

R 

R 

Criterion 

Shrunken 

Shrunken 

Shrunken 

Element 

Weight 

Cross- 

Variable 

Beta 

Weight 

Cross- 

Variable 

Beta 

Welehc 

Cross- 

Variable 

Beta 

Developm. 

val. 

Selected 

Weight 

DeVelopm. 

val. 

Selected 

Weight 

Levelopra, 

val. 

Selected 

Weight 

Using 

Nonverbal 

.39 

.17 

Object  No.® 

-.48 

.57 

.42 

Counting 

.34 

Sounds 

.67 

.52 

Numbers® 

ETST 

-.36 

Analyzing 

Information 

.51 

. 30 

RADO 

-.51 

.66 

.76 

.46 

.57 

CLER 

DDT® 

.46 

-.38 

.35 

.22 

Object  No.® 

.36 

Skill  in 

.46 

.33 

Memory  for 

-.33 

Compiling 

Objects® 

Data 

.54 

.40 

Hidden 

.29 

Patterns® 

Manual 
Control t 

.54 

.39 

Nonsense 

-.61 

Syllogisms® 

.64 

.47 

Cestal t 

-.39 

Precision 

Tools 

Completion® 

.76 

.61 

CLER 

.41 

Skill  Operating 

- 

.37 

.20 

12  (>iestlons® 

.48 

Keyboard  Devices 

.50 

.35 

YRBI 

.36 

Assembling-* 

Disassembling 

Items 

.47 

.30 

Memory  for 
Objects® 

-.47 

Band-arm 

Manipulation, 

Coordination 

.43 

.58 

.70 

.12 

.32 

.48 

Memory  for 
Objects® 

12  Questions® 
MECH 

-.50 

.38 

.39 

.40 

.20 

Memory  for 

.34 

Hand-ear 

Patterns, 
True-false  ® 

Coordination 

.54 

.37 

CLO-LAT® 

-.37 

.62 

.44 

ARl 

.31 

Persuading/ 

.49 

.39 

Inference** 

.67 

Influencing 

.65 

.55 

MECH 

.62 

Others 

.78 

.71 

Password  ® 

-.60 

Vigilance, 

.52 

.31 

YRBI 

-.59 

Continually 

.63 

.40 

12  Questions® 

-.42 

Changing 

.75 

.55 

Recognizing 

-.41 

Details 

Objects  ® 

'Experimental  variable 


TABLE  15 


Predictability  of  Global  Performance  Marks  from  Ratings 
for  the  Common  Job  Elements 


Rating 

R 

Predictor 

Beta  Weight  in 
Final  Composite 

N 

Weight 

Determination 

Expected 

Cross-validation 

Electrician's  Mate 

.70 

.62 

Coping  with  Time 

.71 

20 

.76 

.65 

Ad j . Machines  / Equip . 

-1.02 

.83 

.72 

Assembling,  Disassembling 

.73 

.94 

.88 

Verbal  Communication 

.52 

Personnelman 

.71 

.68 

Using  Written  Materials 

.54 

48 

.79 

.76 

Coping  with  Time 

.38 

Sonar  Technician 

.70 

.64 

Keeping  Up-to-date 

.51 

29 

.79 

.74 

Good  Precision 

.42 

Apprenticeship  Group 

.65 

.63 

Attention  to  Details 

.22 

71 

.77 

.74 

Using  Written  Materials 

.46 

.78 

.75 

Working  with  Distractions 

.21 

Since  about  40  percent  of  the  variance  of  the  global  marks  was 
not  explainable  from  the  Job  element  marks,  other  factors  could 
have  Influenced  the  global  marks — either  as  Independent  effectors 
or  as  moderating  variables.  In  this  respect,  two  possibilities  as 
moderators  were:  (1)  amount  of  time  spent  on  a Job  element,  and 

(2)  the  relative  Importance  of  performance  on  the  Job  element  to 
performance  on  the  total  Job.  This  Is  plausible  because  global  marks 
probably  reflect  not  only  performance  on  the  pertinent  Job  elements, 
but  also  the  Importance  of  the  Individual  elements  for  the  total  Job. 

The  Influence  of  supervisory  ratings  of  these  variables  as 
possible  moderators  of  global  performance  was  Investigated,  using 
composite  variables  formed  from  weighted  performance  marks  for  the 
most  predictive  Job  elements  (Table  15) . The  marks  by  supervisors 
for  time  spent,  and  for  Importance  were  used  to  weight  the  Job 
element  marks  In  accordance  with  three  paradigms: 

1.  Linear  prediction,  as  defined  by  Saunders  (1956).  For  this 
method  either  the  Time  Spent  or  Importance  marks  corresponding  to 
the  Performance  marks  In  Table  15  were  used  together  with  the 
Performance  marks  as  predictors  of  global  performance. 

2.  Moderated  prediction,  as  defined  by  Saunders  (Ibid.).  This 
method  utilizes  as  predictor  sets  the  appropriate  Job  element  marks 
In  Table  15,  either  their  Time  Spent  or  Importance  marks,  and  the 
cross  products  of  all  of  these  variables. 

3.  Weighting  the  Performance  marks  by  Time  Spent  or  Importance 
marks. 

Because  of  the  relatively  small  sample  sizes.  It  was  possible 
that  Increases  In  predictive  accuracy  which  might  be  observed  with 
moderator-variable  techniques  might  result  from  over-fitting  the 
regression  line  Instead  of  tapping  additional,  reliable  predictor/ 
criterion  covariance.  Therefore,  using  the  following  decision  rules 
two  sizes  of  predictor  batteries  were  selected: 

1.  Restricted  predictor  sets  for  moderated  regression  were 
limited  to  the  number  of  variables  that  were  selected  for  the 
unmoderated  runs  (shown  In  Table  15),  l.e.,  four  for  EH,  two  for 
PN,  etc. 


2.  Maximum  sets  were  selected  by  means  of  the  method  for 
selecting  predictor  batteries  for  multiple  regression  described 
previously  In  this  report. 
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Table  16  shows  the  shrunken  ^ of  optimal  batteries  formed  by 
using  Performance  (P) , Importance  (I) , and  Time  Spent  (T)  ratings 
as  predictors.  Column  8 of  Table  16  shows  the  number  of  variables 
selected  for  unmoderated  predictor  batteries.  The  Rs  from  data  sets 
formed  for  three  different  models  are  shown  in  columns  2 through  7. 

The  total  numbers  of  variables  selected  for  the  moderated 
predictor  sets  were  larger  in  all  cases  than  the  number  selected 
for  the  unmoderated  sets.  Computing  global  marks  as  linear  sums 
of  the  raw  job  element  marks,  weighted  by  the  I or  T marks,  typically 
decreased  the  IRs,  with  the  greatest  decreases  being  associated  with 
the  T weights.  For  EMs,  PNs,  and  STs,  maximum  sets  of  both  the 
moderated  techniques  substantially  Increased  the  Rs  computed  from 
unmoderated  regressions,  with  generally  greater  Increases  resulting 
from  moderated  rather  than  linear  prediction.  However,  both  the 
linear  and  the  moderated  Increments  largely  disappeared  for  Restricted 
models.  Weighting  and  moderated-variable  techniques  provided  no 
substantial  Improvement  In  predicting  global  marks  for  UAs. 

On  balance,  the  moderated-variable  techniques  appear  not  to  have 
improved  the  accuracy  with  which  global  marks  could  be  predicted 
from  marks  for  job  elements. 


Accuracy  of  Expert  Judgments  of  Job  Element  Requirements 

Judgments  of  personnel  experts  concerning  the  relative  Importance 
of  attributes  for  jobs  are  usually  the  basis  for  the  Initial  selection 
of  tests  for  a battery.  Thus,  the  battery’s  usefulness  depends 
heavily  on  the  quality  of  the  judgments  underlying  it.  Since  it  is 
well  known  that  opinions  are  sometimes  based  on  unsystematic 
observations  and  personal  predilections,  it  is  pertinent  to  investi- 
gate the  accuracy  of  judgments  of  personnel  experts  concerning  test 
validities.  Median  estimates  by  personnel  experts  of  the  importance 
of  specific  attributes  for  job  performance  of  job  elements  were 
reported  in  previous  research  (Mecham  & McCormick,  1969) . The 
present  study  offered  a unique  opportunity  to  compare  the  accuracy 
of  those  estimates  with  the  validities  actually  found.  For  this 
purpose,  the  median  validity  of  experimental  and  operational  tests 
designed  to  measure  the  same  personal  attribute  (Short-term  Memory, 
Perceptual  Speed,  Perceptual  Closure,  Movement  Detection,  or  Dealing 
with  Concepts/Information)  were  computed  for  all  of  the  common  job 
elements.  The  median  validities  were  correlated  with  the  median 
personal  at tribute- job  element  ratings  computed  by  Mecham  and 
McCormick  (Table  17). 
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TABLE  16 


Shrunken  Cross-validation  Coefficients  for  Optimal  Batteries  of  P,  PI,  and  PT 
Variables  for  the  Most  Predictive  Job  Elements  for  Each  Rating 


(1) 

(2) 

(3) 

(4) 

(5) 

(6) 

(7) 

(8) 

(9) 

(10) 

(11) 

Rating 

Shrunken  R 

Number  of 

Predictors 

Selected 

Sample 

Size 

P 

PI 

PT 

Linear 

(Maxi- 

mum) 

Linear 

(Restric- 

ted) 

Moderated 

(Maximum) 

Moderated  P and 

(Restricted) (Restric- 
ted) 

Linear 

(Maximum) 

Moderated 

(Maxlffltun) 

EM 

88 

71* 

46** 

93 

88 

96** 

90 

4 

6 

6 

27 

PN 

76 

69 

61 

80 

76 

85 

67 

2 

4 

8 

30 

ST 

74 

80 

72 

81 

78 

95** 

86* 

2 

6 

9 

37 

UA 

75 

62* 

60** 

75 

75 

78 

76 

3 

3 

7 

111 

Note,  Decimal  points  omitted  from  cross-validation  coefficients. 


*£  < .05. 

**£  < .01. 


TABLE  17 


Predictive  Validity  of  Median  Personal  Attribute -Job  Element 
Marks  from  Mecham  and  McCormick 


Personal 

Directionality 

Rating 

All 

Attribute 

of  r 

EM 

PN 

ST 

UA 

Ratings 

Movement 

Signed 

47 

-07 

01 

-06 

-05 

Detection 

Absolute  value 

-48 

-33 

-14 

-09 

-06 

Dealing  with 

Concepts/ 

Information 

Signed 

Absolute  value 

-25 

04 

67* 

67* 

-32 

-30 

41 

39 

25 

25 

Closure 

Signed 

-06 

-42 

-20 

53 

04 

Absolute  value 

-40 

37 

-27 

34 

-08 

Perceptual 

Signed 

-41 

-51 

-57 

05 

-29 

Speed 

Absolute  value 

-42 

-24 

-43 

05 

-19 

Short-term 

Signed 

31 

46 

43 

42 

45** 

Memory 

Absolute  value 

-29 

46 

52 

19 

-01 

Note,  Decimal  points  omitted  from  coefficients. 

*£  < .05. 

**£  < .01. 


Both  signed  and  unsigned  values  of  the  validity  coefficients 
were  used  to  compute  the  rs  in  Table  17.  The  lower  rows  of  Table 
17,  based  on  the  unsigned  values,  provided  statistics  which  were 
consistent  with  the  phrasing  in  the  questionnaire  used  by  Mecham 
and  McCormick  (the  experts  were  asked  only  for  the  magnitude  of 
the  relationships,  not  their  directionality).  Although,  in  general, 
the  lower  validities  were  those  based  on  the  unsigned  values,  the 
differences  were  not  substantial.  The  following  comments  refer  to 
the  correlations  of  the  signed  values. 
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The  coefficients  varied  substantially  from  rating  to  rating.  The 
usefulness  of  Dealing  with  Concepts/Information  was  accurately  esti- 
mated for  PNs,  but  the  estimates  were  wide  of  the  mark  for  STs — 
actually  having  negative  validities.  On  the  other  hand,  the  accuracy 
of  estimates  for  Short-term  Memory  tended  to  be  moderate  and  positive 
across  ratings.  The  low  values  for  Perceptual  Speed  were  surprising 
In  view  of  the  extensive  use  of  tests  measuring  this  attribute.  In 
general,  the  estimates  by  experts  of  job  element/personal  attribute 
relationships  were  not  very  accurate.  Only  two  of  the  25  correlations 
were  statistically  significant. 

Whether  or  not  personnel  experts  know  the  personal  attribute 
requirements  of  job  elements  for  positions  In  their  specific  organi- 
zations may  still  be  an  unsettled  question.  However,  these  data  sug- 
gest that  personnel  experts  do  not  accurately  make  such  determinations 
In  the  abstract. 


Comparisons  of  the  Predictive  Accuracy  of  Synthetic-validity  and 
Multiple-regression  Techniques 

By  focusing  on  the  marks  for  the  Individual  job  elements,  synthetic 
validity  theoretically  offers  substantial  increases  in  accuracy  over 
multiple  regression.  For  one  thing.  It  adjusts  estimates  of  job  per- 
formance to  allow  for  task  differences  among  jobs.  Second,  by  restrict- 
ing the  scope  of  the  rating  element.  It  should  decrease  the  amount  of 
halo  effect  In  the  marks. 

The  most  promising  study  of  the  synthetic -validity  technique  pub- 
lished to  date  (Gulon,  1965)  minimized  the  Influence  of  halo  on  ratings 
of  job  elements  by  providing  for  close  personal  supervision  of  the 
raters.  Because  the  supervisory  marks  were  collected  by  mall  In  the 
present  study,  similar  supervision  was  not  possible,  nor  was  it  desir- 
able, from  the  Navy's  standpoint.  If  a questionnaire  technique  were 
found  to  improve  predictability  of  supervisory  marks,  It  might  be 
adaptable  for  operational  use,  whereas  a technique  requiring  close 
supervision  of  the  supervisory  marking  process  probably  would  not  be 
useful  In  the  operational  setting. 

Because  of  the  relatively  small  sample  sizes,  synthetic  validity 
analysis  was  not  carried  out  separately  for  each  of  the  three  techni- 
cal ratings.  Instead  the  data  for  EMs,  PNs,  and  STs  were  combined 
to  form  a Technical  group.  Synthetic-validity  coefficients  were  com- 
puted by  means  of  the  following  procedure;  (1)  Personnel  records 
sorted  Into  Technical,  UA,  and  Total  Group  samples,  each  randomly 
divided  Into  Weight  Determination  and  Alternate  subsamples;  (2)  a 
double  cross-validation  design  was  used  In  which  the  weight  determina- 
tion equations  were  computed,  then  cross  validated  on  their  alternate 
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sample  counterparts,  and  vice  versa;  (3)  for  each  subsample,  the 
set  of  Job  elements,  predictive  of  global  performance,  was  identified 
by  a multiple-regression  technique;  (4)  from  additional  multiple- 
regression  runs,  those  job  elements  were  predicted  by  sets  limited 
to  four  or  fewer  variables;  (5)  equations  for  these  sets  were  used 
to  compute  predicted  ^ scores  for  the  appropriate  subsample,  and 
the  £ scores  were  summed  to  arrive  at  estimated  performances  for 
the  job  elements;  (6)  these  estimates,  in  turn,  were  summed  to 
form  total  predicted  scores  which  were  ordered  and  split  into  highs 
and  lows;  and  (7)  these  types  of  statistics  were  also  computed  by 
multiple  regression,  using  global  performance  as  the  criterion. 
Two-by-two  tables  summarizing  the  synthetic-validity  and  multiple- 
regression  statistics  for  the  cross  validations  are  shown  in 
Table  18.  The  percentage  of  correct  predictions  is  shown  in  the 
two  right-hand  columns  of  the  table. 

Clearly,  synthetic  validity  did  not  result  in  more  accurate 
predictions  than  the  regular  multiple-regression  technique.  For 
both  the  UA  and  the  Total  Group  samples,  the  accuracy  of  synthetic 
validity  was  substantially  lower  than  that  of  multiple  regression. 

For  the  Technical  samples,  synthetic  validity  was  about  as  good, 
overall,  as  multiple  regression.  Thus,  synthetic  validity  does 
not  appear  to  be  promising  as  a technique  for  Improving  the 
predictability  of  job  performance  of  Navy  enlisted  men. 

A probable  reason  for  these  disappointing  results  is  shown  in 
Tables  19  and  20,  which  present  the  intercorrelations  among  the 
supervisory  marks  for  the  common  job  elements  as  they  were  defined 
for  computing  the  synthetic-validity  statistics.  Despite  the 
orthogonality  implied  for  job  elements  from  their  factor-analytic 
origins,  the  performance  marks  were  substantially  intercorrelated. 
Communallty  estimates  for  these  marks  ranged  from  30  to  60  and  were 
about  the  same  magnitude  as  the  job  element /global  mark  correlations. 
Thus,  it  is  apparent  that  halo  effect  was  not  eliminated  by  using 
job  element  marks. 


FINDINGS  AND  CONCLUSIONS 


Prediction  of  Global  Criteria 

1.  About  10  percent  of  the  zero-order  validities  of  experimental 
tests  were  statistically  significant,  most  of  which  were  for  Sonar 
Technician.  No  experimental  test  was  significant  for  more  than  one 
rating. 
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TABLE  18 


Predictive  Accuracy  of  the  Synthetic-validity  and  Multlple-correlaticxi  Methods  for  the  Technical » 
Apprenticeship,  and  Total  Croup  Cross-validation  Subsaaples 


Predicted  Performance  by  Performance  Predicted  Performance  by  Xiyrovement  Over  .Chance 

Subsamples  Means  of  Synthetic  Validity  Rating  Multiple  Regression  Synthetic  Validity  Multiple  Regression 


P to  M 

S 

Apprenticeship 

P to  M 

S 

i 

1 

u n 

14 

10 

Superior 

11 

13 

- 1.81 

21.42 

17 

14 

Poor  to  Mediocre 

21 

11 

i: 

i 

A 

13 

10 

Superior 

10 

13 

5.45 

27.27 

'i 

A 

19 

13 

Poor  to  Mediocre 

22 

10 

Techcilcal 

1 

ii 

10 

12 

Superior 

12 

10 

32.20 

18.64 

W » D • 

27 

10 

Poor  to  Mediocre 

25 

12 

10 

13 

Superior 

9 

14 

28.81 

38.98 

/ 

A 

25 

10 

Poor  to  Mediocre 

27 

9 

Total  Group 


29 

17 

Superior 

28 

18 

40 

29 

Poor  to  Mediocre 

41 

28 

25 

21 

Superior 

23 

23 

42 

25 

Poor  to  Mediocre 

45 

23 

- .86 


11.5 


2.60 


19.29 


TABLE  19 


Intercorrelations  Among  Performance  Marks  for  the  Common  Job  Elements 

for  the  Apprenticeship  Ratings 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

M 

N 

Estimated 

Communallty  Global 

1 Writing  — 
Mats. 

46** 

58** 

49*4c 

68** 

64** 

60** 

57** 

59** 

70** 

3.14 

.99 

71 

59** 

72** 

2 Visual 
Display 

— 

31* 

48** 

31* 

27* 

23 

17 

00 

38** 

3.20 

.79 

66 

30* 

33** 

3 Verb. 
Conini* 

— 

51** 

52** 

66** 

65** 

50** 

63** 

61** 

3.15 

.98 

92 

56** 

52** 

4 Nonprec. 
Tools 

— 

52** 

51** 

49** 

45** 

38** 

51** 

3.29 

.84 

80 

48** 

51** 

5 Routine 
Info. 

— 

74** 

60** 

61** 

67** 

59** 

3.19 

.99 

69 

59** 

60** 

6 Good 
Free. 

— 

88** 

60** 

65** 

64** 

3.26 

.88 

69 

59** 

59** 

7 Attention 
Details 

— 

65** 

70** 

69** 

3.14 

.98 

102 

55** 

65** 

8 Cope 
Time 

— 

58** 

61** 

3.23 

.77 

78 

60** 

54** 

9 Work 
Dlst. 

— 

60** 

3.22 

.90 

84 

59 

64** 

10  Up-to- 
date 

— 

3.09 

.92 

86 

59 

66** 

Note.  Decimal  points  have  been  omitted  from  correlation  coefficients. 

*£  < .05* 

**£  < .01. 


TABLE  20 


lAtercorrelctloos  A«oag  PcrfonMace  tterlia  for  the  Comdh 
Job  ClaMAto  for  Tecbnlcal 


1 2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

21 

22 

n 

SD 

II 

Bs  timmted 
CoMBamlity 

Global 

1 

Nat..  ^ 

46** 

51** 

46**  60** 

48** 

39** 

33** 

61** 

53** 

40** 

56** 

82** 

42** 

62** 

73** 

57** 

68** 

52** 

51** 

58** 

3.14 

.75 

88 

54** 

58** 

j Fict. 

Iteta. 

49** 

41**  42** 

53** 

44** 

30* 

36** 

-01 

60** 

58** 

33* 

27* 

59** 

56** 

32* 

47** 

46** 

28* 

40** 

67** 

3.17 

.78 

58 

43** 

48** 

. Tlmual 
Dlmp. 

— 

27 

51** 

58** 

18 

43** 

28* 

-78** 

55** 

41** 

32* 

60** 

43** 

33* 

44** 

31* 

42** 

38** 

19* 

40** 

3.37 

.69 

52 

42** 

46** 

. Verb. 
Com. 

— 

56** 

37** 

36** 

36** 

46** 

48** 

30** 

46** 

34** 

33** 

40** 

53** 

37** 

42** 

39** 

36** 

56** 

39** 

3.12 

.78 

103 

42** 

41** 

^ ■oorerb. 
^ Smda 

— 

52**  63**  41** 

31* 

38**  36* 

40** 

08 

40** 

37* 

45** 

32* 

48** 

57** 

19* 

40** 

46** 

3.39 

.61 

49 

42** 

50** 

. Amsljxs 

Imfo. 

— 

48** 

58** 

62** 

35** 

63** 

61** 

65** 

60** 

44** 

63** 

56** 

59** 

53** 

51** 

58** 

64** 

3.06 

.84 

67 

53** 

54** 

7 Cosp. 
Imfo. 

— 

28* 

58** 

40** 

34** 

36** 

19* 

29** 

46** 

66** 

32** 

25* 

50** 

15 

35** 

44** 

3.05 

.79 

77 

38* 

44** 

^ ■maprcc. 
® Toolm 

— 

80** 

-09 

55** 

56** 

61** 

38»* 

56** 

33* 

63** 

66** 

65** 

46** 

28* 

58** 

3.45 

.74 

56 

48** 

41** 

^ Free. 
’ Tools 

— 

38** 

50** 

60** 

66** 

37* 

46** 

59** 

65** 

67** 

64** 

54** 

66** 

67** 

3.23 

.88 

48 

50** 

48** 

in 

Device 

— -06 

27* 

00 

60** 

33* 

62** 

55** 

28* 

50** 

54** 

31* 

37** 

3.31 

.86 

55 

42** 

45** 

11 

*4»ip. 

— 

68** 

55** 

68** 

55** 

69** 

58** 

53** 

65** 

58** 

28* 

52** 

3.23 

.82 

57 

52** 

49** 

Assi^le/ 

^ DUMScmble 

- 

59** 

20 

51** 

59** 

57** 

55** 

48** 

64** 

62** 

73** 

3.28 

.82 

53 

46** 

59** 

13 

^ Coord. 

— 

54** 

43** 

52** 

61** 

44** 

56** 

44** 

47** 

41** 

3.37 

.64 

49 

47** 

47** 

. . Isad-cmr 
Coord. 

— 

47** 

66** 

42** 

50** 

63** 

44** 

32* 

33** 

3.21 

.75 

42 

46** 

61** 

ic  Imflmemcc 
Others 

— 

66** 

33** 

38** 

49** 

53** 

42** 

43** 

2.85 

.98 

74 

52** 

39** 

..  Boutime 
Info. 

— 

47** 

45** 

58** 

45** 

55** 

57** 

3.22 

.93 

85 

50** 

57** 

17  ^ 

Precision 

— 

69** 

73** 

57** 

57** 

61** 

3.22 

.83 

87 

51** 

61** 

..  Atta. 
DetsUs 

— 

55** 

43** 

53** 

59** 

3.21 

1.38 

107 

55** 

60** 

19  Titil 

— 

49** 

53** 

58** 

3.35 

.79 

63 

46** 

60** 

“S: 

— 

61** 

60** 

3.15 

.84 

92 

46** 

59** 

Dist. 

— 

54** 

3.05 

.82 

87 

53** 

55** 

22 

^ date 

— 

3.10 

.85 

97 

48** 

65** 

Bote»  Dociaol  points  hsve  been  omitted  from  correlstlon  coeff iclenls. 

*£  < .05. 

*♦£  < .01. 


2.  Generally,  the  operational  tests  were  better  for  predicting 
performance  in  apprenticeship  ratings,  and  the  experimental  tests 
were  better  for  predicting  performance  in  technical  ratings. 

3.  When  multiple  regressions  were  run  for  the  global  marks, 
experimental  variables  added  substantially  to  the  maximum  validi- 
ties of  the  classification  tests.  Increases  in  the  shrunken  valid- 
ity coefficients  were  particularly  high  for  the  three  technical 
ratings. 


4.  Movement  Detection  and  Dealing  with  Concepts/Information 
were  the  most  promising  attributes  for  computerized  measurement. 


Prediction  of  Performance  on  Job  Elements 

1.  The  job  elements  which  were  most  highly  predictable  were 
those  which  were  obviously  important  and  central  to  the  duties  of 
particular  ratings.  Thus,  for  PNs  these  job  elements  were  Writing, 
Verbal  Communication,  Influencing  Others,  and  Communicating  Routine 
Information;  for  STs,  Visual  Display  was  highly  predictable. 

2.  Operational  variables  were  most  effective  for  predicting 
on-job  performance  of  UAs.  The  significant  zero-order  correlations 

of  test  scores  with  performance  on  job  elements  for  UAs  were  apparently 
a result  of  the  strong  "g"  factor  in  the  operational  variables.  In  the 
technical  ratings,  the  most  effective  predictors  for  job  elements  were 
experimental  tests.  The  best  and  most  consistent  of  these  were  compu- 
terized measures. 

3.  There  was  little  or  no  evidence  of  consistency  of  job  element 
characteristics  across  ratings.  The  variables  having  significant 
zero-order  validities  for  a job  element  in  one  rating  in  general  did 
not  have  significant  validities  for  the  same  element  in  other  ratings. 

4.  Halo  effect  was  not  eliminated  by  using  job  element  marks. 


Predictability  of  Global  Marks  From  Job  Element  Marks 

In  the  technical  ratings,  slight  increases  in  the  accuracy  of 
predicting  global  marks  from  job  element  marks  were  associated  with 
the  use  of  Time  Required  and  Importance  variables  as  moderators. 
However,  the  increases  seemed  to  result  from  capitalization  on 
chance  relationships. 
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Accuracy  of  Expert  Judgments  of  the  Personal  Attribute  Requirements 
for  Performance  of  Particular  Job  Elements  * 

Generally  low  correlations  were  found  between  empirically-derived 
estimates  of  Importance  of  personal  attributes  for  particular  job 
elements  and  similar  estimates  based  on  the  judgments  of  personnel 
experts.  Correlations  of  these  estimates  were  statistically  signifi- 
cant for  only  two  of  the  25  personal  attribute/ job  element  linkages. 
Estimates  for  Short-term  Memory  were  the  most  accurate,  and  those 
for  Dealing  with  Concepts/Information  were  accurate  for  some  ratings 
and  Inaccurate  for  others. 

These  findings  strongly  Imply  that  personnel  experts  cannot  make 
accurate  determinations  of  attribute  requirements  for  job  elements 
In  the  abstract.  Whether  or  not  they  can  make  these  determinations 
for  positions  with  which  they  have  first-hand  familiarity  remains  to 
be  seen. 


Comparisons  of  the  Predictive  Accuracy  of  Synthetic-validity  and 
Multiple-regression  Techniques 

Synthetic  validity  was  generally  not  as  accurate  as  multiple 
regression  for  predicting  job  performance.  A major  reason  for  the 
low  accuracy  of  the  synthetic-validity  technique  was  the  large  halo 
effect  In  the  data. 


RECOMMENDATIONS 


1.  Since  measures  of  Closure  had  significant  relationships  to 
the  job  performance  of  personnel  In  all  four  ratings/rating  groups 
and  had  both  positive  and  negative  beta  weights,  a Closure  test 
would  appear  to  be  highly  suitable  for  a battery  designed  to  predict 
on-job  performance.  A measure  of  Closure  should  be  Included  In  any 
classification  battery  designed  to  predict  supervisory  ratings  of 
on-job  performance.  At  present,  a paper-and-pencll  test  of  Closure 
would  be  just  as  effective  as  a computerized  one,  and  would  be  less 
expensive  to  administer. 

2.  Skills  In  visual  processing  and  Interpretation,  In  addition 
to  those  measured  by  the  operational  classification  tests,  appear  to 
be  very  Important  for  on-job  performance  of  STs.  In  the  present 
research,  measurement  of  these  skills  by  computerized  tests  resulted 
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in  Increases  of  predictability  substantial  enough  to  provide 
practical  Improvement  in  the  selection  of  STs.  However,  the 
findings  should  be  replicated  on  a sample  of  at  least  100,  and 
preferably  200,  STs. 

The  operational  implementation  of  computerized  selection 
testing  will  depend  on  the  magnitude  of  the  tradeoff  between  the 
improvements  in  predictability  and  the  cost  Increases  associated 
with  computerized  testing.  Information  from  an  additional  study 
is  required  to  accurately  evaluate  these  qualities. 

3.  The  research  should  be  extended  to  investigate  whether 
Increases  in  validities  can  be  achieved  in  any  other  rating  (such 
as  Operations  Specialist)  where  interpretation  of  visual  displays 
on  screens  and  scopes  is  Important. 

4.  Since  marks  for  only  one-thirteenth  of  the  personal  attributes 
described  by  Mecham  and  McCormick  were  examined  in  the  present  research, 
it  is  unlikely  that  all  of  the  attributes  which  are  important  for  on- 
job  performance  were  included  in  the  study.  The  substantial  improve- 
ments in  predictability  found  from  the  present  relatively  restricted 
sampling  suggest  that  the  present  attributes,  or  others  which  have 

been  defined,  may  provide  useful  selectors  for  ratings  which  are  not 
now  being  accurately  predicted.  In  this  effort,  the  Mecham  and 
McCormick  study  could  serve  as  a useful  guide  for  selecting  personal 
attributes  which  would  be  appropriate  for  the  major  job  elements  of 
the  ratings. 
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APPENDIX 


POSITION  ANALYSIS  QUESTIONNAIRE  (PAQ) 
DIRECTIONS 


Your  help  is  needed  for  a research  project  involving  the  performance  of  enlisted  Navy 
personnel.  This  questionnaire  is  the  last  step  in  the  evaluation  of  an  extensive  series  of 
tests  which  were  given  to  only  a small  number  of  men,  so  it  is  most  important  that  you  complete 
it.  The  questionnaire  contains  items  to  be  used  to  describe  the  job  held  by  the  person  whose 
name  is  in  the  upper  left  comer  of  this  page.  This  information  is  to  be  used  for  RESEARCH 
PURPOSES  ONLY,  and  will  not  affect  any  individual  involved  in  this  study. 

Please  complete  the  questionnaire  carefully  as  follows: 

1.  Using  the  first  column  on  the  right  (T),  give  the  approximate  percentage  of  time  spent  on 
every  element  included  in  the  man’s  job.  Many  of  the  elements  in  the  questionnaire  will  not 
be  included  in  the  man’s  job  and  you  should  put  a dash  (-)  next  to  those  elements. 

2.  In  the  second  column  (I),  rate  the  importance  of  each  element  included  in  the  man’s  job  to 
overall  job  performance. 

3.  In  the  last  column  (P) , rate  his  overall  job  performance  on  each  element  in  his  job. 

It  is  important  that  all  questions  be  answered  carefully.  Be  sure  to  rate  this  man  for 
every  element  which  is  part  of  his  job.  Each  job  element  describes  some  general  work  activity, 
work  condition,  or  job  characteristic.  In  most  cases  examples  are  given  to  illustrate  the 
’’central  idea”  of  the  job  element.  However,  these  examples  are  intended  only  to  help  illustrate 
the  idea  and  include  only  a few  of  the  possible  examples  that  could  characterize  the  Job  element. 

EXAMPLES: 


Code  Importance  to 
This  Job  (I) 
— Does  not  apply 

1 Very  minor 

2 Low 

3 Average 

4 High 

5 Extreme 


ELEMENTS 


A.  Estimating  the  size  of  objects  without  direct 
measurement. 

B.  Writing  or  dictating  letters,  reports, 
instructions,  technical  manuals,  etc. 

EXPLANATION:  The  man’s  job  does  not  contain  the  activity  in  Example  A,  so  dashes  are  used.  In 
Example  B,  he  does,  however,  write  letters  and  reports  between  1/10  and  1/3  of  the 
time.  They  are  of  average  importance  to  his  job,  and  he  prepares  them  as  well  as 
the  middle  40%  of  personnel  in  his  rate. 


Code 

Amount  of  Time  (T) 

1 

Does  not  apply  (or 
is  very  incidental) 
Under  1/10  of  the  time 

2 

Between  1/10  and  1/3 

3 

of  the  time 
Between  1/3  and  2/3 

4 

of  the  time 

Over  2/3  of  the  time 

5 

Almost  continually 

Compared  to  other  men  of  the  same 
pay  grade,  how  would  you  rank  this 
man  in  the  performance  of  this 

task?  (P) 

Does  not  apply 

1 Lowest  S% 

2 Next  Lowest  25% 

3 Middle  40% 

4 Next  Highest  25% 

5 Highest  5% 


RATINGS 

T - I — P — 


T I 3 P 3 
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Code 

Amount  of  Time  (T) 

Does  not  apply  (or 

is  very  incidental) 

1 

Under  1/10  of  the  time 

2 

Between  1/10  and  1/5 

of  the  time 

3 

Between  1/3  and  2/3 

of  the  time 

4 

Over  2/3  of  the  time 

5 

Almost  continually 

Code 

Importance  to 
This  Job  (I) 

— 

Does  not  apply 

1 

Very  minor 

2 

Low 

3 

Average 

4 

High 

5 

Extreme 

Compared  to  other  men  of  the  same 
pay  grade » how  would  you  rank  this 
man  in  the  performance  of  this 

task?  (P) 

— Docs  not  apply 

1 Lowest  5% 

2 Next  Lowest  25% 

3 Middle  40% 

4 Next  Highest  25% 

5 Highest  5% 


A.  SOURCES  OF  JOB  INFORMATION 


Rate  each  of  the  following  sources  of  job  in- 
formation in  terms  of  amount  of  time  spent 
using  it,  its  importance  to  the  job,  and  this 
person's  proficiency  in  its  use. 


1.  Written  materials  (manuals, 
reports,  office  notes,  job 
instructions,  notices  and 
other  written  instructions.) 


T I P 


T I P 


2.  Pictorial  materials  (pic- 
tures or  picture- like 
materials  used  as  sources 
of  information,  for  example, 
drawings,  blueprints,  dia- 
grams, maps,  tracings, 
photographic  films) 

3.  Visual  displays  (dials, 
gauges,  signal  lights, 
radar/sonar  scopes,  clocks, 
etc. ) 


4.  Features  of  nature  (ocean 
surface  conditions,  geolog- 
ical samples,  underwater 
T I P topography,  cloud  forma- 

tions, and  other  features 
of  nature  which  are  ob- 
served or  inspected  to  pro- 
vide information) 


T I P 


T I P 


T I P 


5.  Behavior  (observing  the 
actions  of  people;  for 
example,  in  teaching,  super- 
vising or  counseling,  where 
this  behavior  is  a source  of 
job  information) 

6.  Spoken  verbal  sources  (ver- 
bal instructions,  orders, 
requests,  conversations, 
interviews,  discussions, 
formal  meetings;  consider 
only  verbal  communication 
which  is  relevant  to  Job 
performance) 

7.  Non-verbal  sounds  (for  ex- 
ample, noises,  engine  sounds, 
sonar,  whistles,  signals, 
horns) 


8.  Depth  perception  (judging 
the  distance  from  the  ob- 
server to  the  objects,  or 

the  distances  between  ob-  7 j p 

jects  as  they  are  posi-  - 

tioned  in  space,  as  in  op- 
erating a dentist's  drill, 
handling  and  positioning 
objects,  etc.) 

9.  Sound  pattern  recognition 
(recognizing  different  pat- 
terns, or  sequences  of 

sounds;  for  example,  those? I P 

involved  in  Morse  Code, 
heart  beats,  engines  not 
functioning  correctly) 


B.  ESTIMATING  ACTIVITIES 

In  this  section  are  various  operations  in- 
volving estimation  or  judging  activities. 
In  each  item  consider  activities  in  which 
the  worker  uses  any  or  all  of  the  senses; 
for  example,  sight,  hearing,  touch. 

10.  Estimating  speed  of  moving 
objects  (estimating  the 
speed  of  moving  objects  or 
materials  relative  to  a 

fixed  point  or  to  other  T I P_ 

moving  objects;  for  exam- 
ple, the  speed  of  vehicles 
materials  on  a conveyor 
belt,  relative  speed  of 
other  ships) 

11.  Estimating  speed  of  pro- 
cesses (estimating  the 
speed  of  on-going  pro- 
cesses or  a series  of 

events  while  they  are  T I P_ 

taking  place;  for  example, 
assembly  operations, 
charging  batteries,  pre- 
paration of  records  and 
reports  to  meet  deadlines) 

12.  Inspecting  (inspecting 
products,  objects,  ma- 
terials, etc.,  either 
one's  own  workmanship  or 

that  of  others,  in  terms  T I P^ 

of  established  standards; 
for  example,  identifying 
defects,  classifying  by 
grade) 
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Code  Amount  of  Time  fT) 

Code  Importance  to 

_ Does  not  apply  (or 

This  Job  (I) 

is  very  incidental) 

— Does  not  apply 

1 Under  1/10  of  the  time 

1 Very  minor 

2 Between  1/10  and  1/3 

2 Low 

of  the  time 

3 Average 

3 Between  1/3  and  2/3 

4 High 

of  the  time 

5 Extreme 

4 Over  2/3  of  the  time 

5 Almost  continually 

13.  Estimating  size 

(estimat- 

ing  the  dimensions  of  ob- 
jects  without  direct  mea» 

T I P surement,  including  length, 

thickness,  etc;  for  example, 
estimating  the  physical  site 
of  a 3onar  contact) 

14.  Estimating  time  (estimate 
ing  the  time  required  for 
past  or  future  events  or 
work  activities;  for  exam- 

T I P pie,  judging  the  amount  of 

time  to  make  a delivery, 
estimating  the  time  re- 
quired to  service  a worn 
machine  part  or  piece  of 
equipment) 

C.  INFORMATION  PROCESSING  ACTIVITIES 

In  this  section  are  various  human  operations 
involving  the  "processing"  of  information  or 
data.  Rate  each  item  on  all  three  scales. 

15.  Analyzing  information  or 
data  (for  the  purpose  of 
identifying  underlying 

T I P principles  or  facts  by 

breaking  down  information 
into  component  parts;  for 
example,  interpreting  re- 
ports or  diagnosing  mechan- 
ical disorders) 

16.  Compiling  (gathering, 
grouping,  classifying,  or 
in  some  other  way  arrang- 
ing information  or  data 
in  some  meaningful  order 

T 1 P or  form;  for  example,  pre- 

paring reports  of  various 
kinds,  filing  correspond- 
ence on  the  basis  of  con- 
tent, selecting  particular 
data  to  be  gathered) 

17.  Coding/decoding  (coding  in- 
formation or  converting 
coded  information  back  to 
its  original  form;  for  ex- 
ample, ’’reading"  Morse 

T I P Code,  translating  foreign 

languages,  or  using  other 
coding  systems  such  as 
shorthand,  mathematical 
symbols,  computer  languages, 
drafting  symbols,  stock  cata- 
log numbers) 


Compared  to  other  men  of  the  same 
pay  grade,  how  would  you  rank  this 
man  in  the  performance  of  this 
task?  (P) 

— 

Does  not  apply 

1 

Lowest  5% 

2 

Next  Lowest  25% 

3 

Middle  40% 

4 

Next  Highest  25% 

5 

Highest  5% 

D.  WORK  OUTPUT 

Use  of  devices  and  equipment.  Consider  in 
this  category  those  powered  devices  which  are 
used  to  move  or  modify  work  pieces,  materials, 
products,  or  objects.  They  are  manually  con- 
trolled or  directed  devices  using  an  energy 
source  such  as  electricity,  compressed  air, 
fuel,  hydraulic  fluid,  etc.,  in  which  the 
component  part  which  accomplishes  the  modifi- 
cation is  hand-held;  such  as  a dentist’s  drill, 
welding  equipment. 

18.  Non-precision  tools/ 

instruments  (tools  or  in- 
struments powered  by  the 
user  to  perform  operations 
not  requiring  great  accu- 
racy or  precision;  for  ex-  T I P 

ample,  hammers,  wrenches, 

trowels,  knives,  scissors, 
chisels,  putty  knives, 
strainers,  hand  grease  guns; 
do  not  include  long-handle 
tools  here) 

19.  Precision  tools/instruroents 
hand-held  powered  tools  or 
instruments  used  to  perform 
operations  requiring  great 

accuracy  or  precisionV  Jen-  T I P 

tist's  drills,  soldering 
irons,  welding  equipment, 
saws,  used  for  especially 
accurate  or  fine  work) 

20.  Drawing  and  related  devices 

(instruments  or  devices  used 
in  plotting,  sketching,  il- 
lustrating, drafting;  for  T I P 

example,  pens,  pencils,  draw- 
ing instruments,  drafting 

equipment;  do  not  include 
measuring  instruments  here) 

21.  Keyboard  devices  (type- 
writers, adding  machines,  T I P 

calculators,  keypunch 

machines) 

22.  Hand-operated  controls  (con- 
trols operated  by  hand  or 

arm  for  making  frequent , T I P 

but  not  continuous,  adjust- 
ments; for  example,  hand 
controls  on  a crane  or  bull- 
dozer, helm  of  ship) 
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Code  Amount  of  Time  (t) 

Does  not  apply  (or 

is  very  incidental) 

1  Under  1/10  of  the  time 
I 2 Between  1/10  and  1/3 

j of  the  time 

I 3 Between  1/3  and  2/3 

i of  the  time 

: 4 Over  2/3  of  the  time 

I S Almost  continually 


Code 

Importance  to 
This  Job  (I) 

— 

Does  not  apply 

1 

Very  minor 

2 

Low 

1 3 

Average 

i 4 

High 

1 5 

Extreme 

Compared  to  other  men  o^  the  same 
pay  grade,  how  would  you  rank  this 
man  in  the  performance  of  this 

task?  (P) 

— Does  not  apply 

1 Lowest  5\ 

2 Next  Lowest  25% 

3 Middle  40% 

4 Next  Highest  25% 

5 Highest  5% 


23.  Setting  up/adjusting  (ad- 
justing, calibrating, 
aligning  and/or  setting  up 
of  machines  or  equipment; 

T I P for  example,  setting  up  a 

lathe  or  drill  press,  ad- 
justing an  engine  carbure- 
tor, adjusting,  calibrating, 
and  aligning  electric  cir- 
cuitry) 


E.  COKWUNICATION  WITH  OTHER  PERSONS 

This  section  deals  with  different  aspects  of 
interactions  between  people  involved  in  vari- 
ous kinds  of  work. 

29.  Persuading  (dealing  with 

others  in  order  to  influence 

them  toward  some  action  or  T I P 

point  of  view) 


24.  Assembling/disassembling 
(either  manually  or  with 
the  use  of  hand  tools 
putting  parts  of  compon- 

T 1 P ents  together  to  form  more 

complete  items,  or  taking 
apart  or  disassembling 
items  into  their  component 
parts) 

25.  Finger  manipulation  (mak- 
ing careful  finger  move- 
ments in  various  types  of 
activities;  for  example, 
fine  assembly,  use  of  pre- 

T I P cision  tools,  repairing 

watches,  use  of  writing 
and  drawing  instruments, 
operating  keyboard  devices; 
usually  the  hand  and  arm 
are  not  involved  to  any 
great  extent) 


26.  Hand-arm  manipulation  (the 
manual  control,  or  manipu- 
lation of  objects  through 

T I P hand  and/or  aim  movements, 

which  may  or  may  not  re- 
quire continuous  visual 
control;  for  example,  re- 
pairing automobiles) 

27.  Limb  movement  without  vis- 
ual control  (movement  of 
arms  or  legs  from  one  posi- 
tion to  another  without  the 

T I P use  of  vision;  for  example, 

reaching  for  controls  with- 
out looking,  playing  a 
musical  instrument,  touch 
typing) 


28.  Hand-ear  coordination  (the 
coordination  of  hand  move- 
ment with  sounds  or  instruc- 

T I P tions  that  are  heard;  for 

example,  tuning  radio  re- 
ceivers, adjusting  sonar 
equipment) 


30.  Instructing  (the  teaching 
of  knowledge  or  skills, 
either  in  an  informal  or 
formal  manner,  to  others;  T 
for  example,  an  A-School 
instructor,  a journeyman 
teaching  an  apprentice) 

31.  Interviewing  (conducting 
interviews  directed  to- 
ward some  specific  objec-  T 
tlve;  for  example,  inter- 
viewing for  job  assign- 
ments) 

32.  Routine  information  ex- 

change (the  giving  and/or 
receiving  of  information 
of  a routine  or  simple  T 

nature;  for  example,  re- 
ceptionist, information 
counter  clerk) 

33.  Writing  (for  example, 

writing  or  dictating 
letters,  reports,  in-  T 

structions,  technical 
manuals) 

F.  TYPES  OF  JOB-REQUIRED  PERSONAL  CONTACT 

This  section  lists  types  of  individuals  with 
whom  the  roan  must  have  personal  contact  in 
order  to  perform  his  job.  Consider  personal 
contact  not  only  with  personnel  within  the 
command  but  also  with  personnel  from  other 
commands,  if  contact  with  them  is  part  of 
the  job. 

34.  Supervisors  (those  person- 
nel who  have  immediate  re- 
sponsibility ^or  a work 

group;  for  example,  Divi-  T I P__ 

sion  Chief  Petty  Officers, 
office  managers) 

35.  Clients,  counselees, 

trainees  T I P 
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Code 

Amount  of  Time  (T) 

— 

Does  not  apply  (or 
is  very  incidental) 

1 

Under  1/10  of  the  time 

2 

Between  1/10  and  1/3 
of  the  time 

3 

Between  1/3  and  2/3 
of  the  time 

4 

Over  2/3  of  the  tune 

5 

Almost  continually 

Code 

Importance  to 
This  Job  (I) 

— 

Does  not  apply 

1 

Very  minor 

2 

Low 

3 

Average 

4 

High 

5 

Extreme 

Coii^arecl  to  other  men  of  the  same 
pay  grade,  how  would  you  rank  this 
nan  in  the  performance  of  this 

task?  (P) 

— Does  not  apply 

1 Lowest  S\ 

2 Next  Lowest  2S% 

5 Middle  40% 

4 Next  Highest  2S% 

5 Highest  5% 


56.  Frustrating  situations  (job 
situations  in  which  attempts 
to  deal  with  problems  or  to 
T I P achieve  job  objects  are  ob- 

structed or  hindered,  and 
may  thus  contribute  to 
frustration  on  the  part  of 
the  worker) 

G.  JOB  DEMANDS 

This  section  lists  various  types  of  demands 
that  the  job  situation  may  impose  upon  the 
worker,  usually  requiring  that  he  adapt  to 
these  in  order  to  perform  his  work  satisfac- 
torily. 


H.  OVERALL  PERFORMANCE  EVALUATION 

43.  Considering  all  aspects  of 
the  job,  how  good  is  this 
man's  performance? 

(1)  Lowest  5% 

(2)  Next  lowest  10% 

(5)  Next  lowest  15% 

(4)  Middle  40Z  T I P 

(5)  Next  highest  15% 

(6)  Next  highest  10% 

(7)  Highest  5% 


T I P 


T I P 


T I P 


T I P 


T I P 


T I P 


57.  Precision  (need  to  be  more 
than  normally  precise  and 
accurate) 

58.  Attention  to  detail  (need 
to  give  careful  attention 
to  various  details  of 
one's  work,  being  sure 
that  nothing  is  left  un- 
done) 

59.  Vigilance:  continually 

changing  events  (need  to 
be  continually  aware  of 
variations  in  a contin- 
ually or  frequently 
changing  situation;  for 
example,  continually 
watching  frequently 
changing  dials,  driving  in 
traffic,  controlling  air- 
craft traffic) 

40.  Time  pressure  of  situation 
(urgent  time  deadlines, 
rush  jobs) 

41.  Working  under  distractions 
(telephone  calls,  inter- 
ruptions, disturbances 
from  others) 

42.  Updating  job  knowledge 
(keeping  job  knowledge 
current,  being  informed  of 
new  developments  related 
to  the  job) 
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DISTRIBUTION  LIST 


Assistant  Secretary  of  the  Navy  (Manpower  and  Reserve  Affairs)  (2) 
Assistant  Secretary  of  the  Navy  (Research  and  Development) 

Chief  of  Naval  Operations  (0P-103B) 

(OP-914) 

(OP-964) 

(OP-987^0) 

Chief  of  Naval  Personnel  (Pers-1) 

(Pers-2) 

(Pers-5) 

(Pers-lOc) 

Chief,  Bureau  of  Medicine  and  Surgery  (Code  713) 

Chief  of  Naval  Research  (Code  450)  (4) 

(Code  458)  (4) 

Chief  of  Naval  Technical  Training  (Code  016) 

Chief  of  Naval  Education  and  Training  (Code  N2) 

(Code  N5) 

(Code  N-33) 

Chief  of  Naval  Material  (NMAT-0344) 

(NMAT-0324) 

(NMAT-035) 

Chief  of  Naval  Training  Support  (Code  21) 

Chief  of  Naval  Reserve  (Code  3055) 

Commander  Training  Command,  U.  S.  Atlantic  Fleet  (Code  N3A) 

Commander  Navy  Recruiting  Command  (Code  015) 

Commander,  Naval  Training  Center,  San  Diego 
Commander,  Naval  Air  Systems  Command  (AIR-413E) 

(AIR-413C) 

(AIR-415C) 

Commander,  Naval  Sea  Systems  Command  (047C2) 

(Technical  Library) 

Commanding  Officer,  U.  S.  Naval  Amphibious  School,  Coronado 
Commanding  Officer,  Naval  Health  Research  Center,  San  Diego 
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